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Abstract 

We investigate the Large Deviations properties of bootstrapped empir- 
ical measure with exchangeable weights. Our main result shows in great 
generality how the resulting rate function combines the LD properties of 
both the sample weights and the observations. As an application we re- 
cover known conditional and unconditional LDPs and obtain some new 



Keywords Large deviations, Exchangeable bootstrap 

1 Introduction and main results 

We say that a sequence of Borel probability measures (P n ) n >i on a topological 
space y obeys a Large Deviation Principle (hereafter abbreviated LDP) with 
rate function I if / is a non-negative, lower semi-continuous function defined on 
y such that 



- inf I(y) < liminf -logP n {A) < limsup - logP n (A) < - inf I(y) 

y£A° n^oo n n^oo U yGA 

for any measurable set A C y, whose interior is denoted by A° and closure by 
A. If the level sets {y : I(y) < a} are compact for every a < oo, I is called 
a good rate function. With a slight abuse of language we say that a sequence 
of random variables obeys an LDP when the sequence of measures induced by 
these random variables obeys an LDP. For a background on the theory of large 
deviations, see Dembo and Zeitouni [8] and references therein. 
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Our framework is the following: We are given a triangular array ((W")i<j< ra )„>i 
of R + -valued random variables defined on a probability space (f2, A, P) and such 
that 

(HI) For every n > 1 we have Y^i=i ^7 = 71 • 

(H2) For every n > 1 the vector (W™, . . . , W") is n-exchangeable i.e. for ev- 
ery element a of & n the set of permutations of {1, ...,n} the vectors 
{W{\ . . . , W") and , . . . , W; l (n) ) have the same distribution. 

We shall assume that the W"'s have some LD properties to be detailed below. 
We are further given a triangular array ((a:")i<i<n)n>i of elements of a Polish 
space (E,cZe) such that 

n 

= ^/^GAA(S) (1-1) 

i=l 

where ^> stands for weak convergence in the space of Borel probability measures 
on S. Let us recall that /x™ — ^ if and only if for every real- valued, bounded and 
continuous application / defined on £ we have J s f (x) /j, n (dx) —> J E f (x) fi(dx) . 
Our goal in the present paper is to investigate the LD properties of 

n 

r = -Vf^». (i.2) 

So far LD properties of families of randomly weighted empirical measures like 
(1.2) have been established in the particular case of independent and identi- 
cally distributed W±, . . . ,W™, see e.g. [15, 7, 23]. Results on the LD proper- 
ties of some particular cases of C" as considered here, i.e. with exchangeable 
Wi , . . . , W™ are available but their proofs rely on the definition of the chosen 
sampling weights, see e.g. [2, 7, 6]. Hence, the present paper gives the first 
derivation of the LD properties of the family of empirical measures C n under 
the natural assumptions (H1-H2). The interest in considering families rather 
than particular cases lies on the fact that only an upper level of abstraction can 
reveal the mechanisms that really enter into play. 

Here we shall consider C n from the bootstrap point of view and adopt its 
vocabulary [13, 18, 1, 16]. In 1979, in a landmark paper, Efron proposed 
the following idea: When given a realization x™ , • ■ • , x™ of random variables 
X", . . . ,X™ one can easily obtain "additional data" by sampling independent 
and i S™=i ^"-distributed random variables X^, . . . , X^, .... It amounts to 
sample with replacement from an urn which composition is described by — • 
Often, this is computationally cheap and theoretical studies are available to as- 
sess the quality of the distribution of H{^- ^Xt) m approximating the 
distribution of H(^Y^=i^xv-) which make it all worthwhile. A rich litera- 
ture started flourishing on the ground of this idea. It was soon noticed that 
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the preceding procedure is not the only possible one and that it can be gen- 
eralized so that - Y^i=i W"5 x n under conditions (H1-H2) is the right object 
to be considered. For example, Efron's bootstrap corresponds to (W™, . . . , W") 
distributed according to a Multinomial law. The literature on the subject devel- 
oped into two complementary directions: "conditional" results where x™, . . . , x™ 
are fixed observations filling some conditions and H(— X)"=i W™5 x n) is consid- 
ered and " unconditional" results where the x™ , . . . , x™ are allowed to fluctuate 
and Y^7=i W™8x") is considered instead. 

Classically the bootstrap scheme is said to be efficient when it mimics the 
behavior of H (i Y^h=x Sxv-) and one distinguishes between "conditional effi- 
ciency" and " unconditional efficiency" . For example, Praestgaard and Wellner 
investigated the Central Limit behavior of C n in [24] ; they gave a necessary and 
sufficient condition on the second moments properties of the W™ which works 
for both conditional and unconditional efficiency. Later, Hall and Mammcn 
studied the efficiency of the bootstrap schemes in the Edgeworth expansions 
at the second order [19]. There, conditions on the fourth order cumulants of 
the weights are required. In a similar context, Wood already showed in [28] 
that Efron's bootstrap is efficient to mimic the empirical mean in the moderate 
deviations regime for observations satisfying the Cramer condition but not for 
heavier tails. 

Following Barbe and Bertail [1], we say that a bootstrap scheme is LD- 
efficient when the bootstrapped empirical measure has the same LD properties 
as the original empirical measure. It is a very strong property: thinking of 
percentile bootstrap's confidence intervals, LD-efficiency says that the relative 
coverage accuracy tends to 1 exponentially fast. As an application of our general 
approach, we will be able to discuss both conditional and unconditional LD- 
efficiency for many classical choices of ((W")i<i<„)„>i and/or ((X™)i<;<„) n >i- 
Actually we go further in the sense that as an application of our approach we 
obtain LD results that are new in the literature like e.g. an unconditional LDP 
for Efron's bootstrap and conditional and unconditional LDP's for iid weighted 
bootstrap and fc-blocks bootstraps. 

Now let us describe our results more precisely. To this end we need to 
introduce some more notations. We consider 



the so-called Wasserstein space of order 1 on R+. Both M 1 1 (R + ) and W 1 (R+) 
are endowed with the Wasserstein- 1 distance 




a subset of 





inf 

ireC(p, 7 ) 
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where C(p, 7) is the subset of Mi(R+ x R + ) of couplings of p and 7 i.e. the set 
of Borcl probability measures ir on R + x R + such that their first marginal tti is 
p and second marginal 7T2 is 7. In addition to (H1-H2) we shall assume that 
the ((W7*)i<i< n ) n >i are such that 

(H3) The sequence (5" = — J27=i Sw n )n>i satisfies an LDP on Af/-(R+) en- 
dowed with the Wi u distance with good rate function I w . 

In Section 7 we prove that (H3) is valid for a broad collection of sampling 
weights. Actually we shall see that the LD properties of (£ n ) n >i directly follow 
from the LD properties of 



1 n 

i=l 

on the set 

M{(R+ x E) = J p e Ah(R+ x E) : f 



xpi(dx) = 1 



endowed with the distance 

A(p,7) = Wl,|.|(pi,7i) +Pbl,s(p,i) 

where 



/(x)^(da;) - / /(x) 7 (dx) 



/3bl,s(p,j)= sup 

/ec b (E + xs) 

l/lloo+||/|| L , 4 <l 

is the so-called dual-bounded Lipschitz metric on Mi(R + x E) and as usual 
I l/l U = sup |/(x)|, ||/||l,5= sup — 

S is a metric on R + x E compatible with the product topology and C;,(R+ x E) is 
the set of real- valued, bounded and continuous applications defined on R + x E. 
For any two probabilities p, v on a measurable space (E, £) we denote by 

H{u\p) = { ^ l0g cfe if 

+c» otherwise 

the relative entropy of v with respect to p. To any p(dw, dx) £ Mi(R+ x E) we 
associate p x (dw) € Mi(R + ) (resp. p w (dx) G Mi(E)) a stochastic kernel which 
is the conditional distribution of the first (resp. second) marginal of p given the 
second (resp. first). We summarize this by p(dw,dx) = p x {dw) <g> p2(dx) (resp. 
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p(dw, dx) = pi(dw) <g> p w {dx)). If z/,7 € Mi(K + x E) are such that V\ = 71 = 
(resp. ^2 = 72 = then 

#H7)= / #O w |7»)0(cM (1-3) 

(resp. H{yYf) = H(i> x \-f x )9(dx)), see Lemma 1.4.3 in [12]. Our first result is 
the following 

Theorem 1.1. The sequence (V n )„>i satisfies an LDP on A / I}(R+ x E) en- 
dowed with the distance A with good rate function 



H{p\p 1 ®p)+I w {p 1 ) if p 2 = p 
+00 otherwise. 



The reason for working on _A/f}(IR+ x E) endowed with the distance A is to make 
the following map 

F : MlQSL+xY.) -> M X (E) 

p(dw, dx) h-> / R wp(dw, dx) 

well-defined and continuous once Mi(E) is endowed with the weak convergence 
topology. By contraction (see Theorem 4.2.1 in [8]) an LDP for (C n ) n >i easily 
follows from Theorem 1.1 

Corollary 1.1. The sequence (C n ) n >i satisfies an LDP on Mi(E) endowed 
with the weak convergence topology with good rate function 

K(y;n) = inf J{p;\i) 

p:F{p)=u 

inf ( J H(p x \ Pl )p(dx)+I w (p 

We shall often see in applications that I w (v) = H(y\£) for some £ <G Mi(K + ) 
such that 

Af(a) = log f e ax i{dx) < 00 (1.4) 
for every ael. We denote by 

At(x) = sup {ax - A f (a)} . (1.5) 

the Fenchel-Legendre transform of Aj . In this particular case we obtain a gen- 
eralization of Kullback inequality 

Lemma 1.1. If L w (v) = H(v\£) for some £ G Mi(R+) then for every is, p € 
Mi(E) we /iawe 

£(,;„)>( / S A Kf(-))^) (1 . 6) 
+00 otherwise. 
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Moreover, assuming that (1-4) holds for every a G R, the preceding inequality 
turns out to be an equality for every i>, p G Mi(£) if and only if 

lim A f (a) = and lim A c (ce) = +00. (1-7) 

Remark that the right hand side of the inequality (1.6) is the rate function 
obtained in case of iid weights in [7] . 

Next we allow the x"'s to fluctuate and consider a triangular array ((X")i<;<„)„>i 
of S-valued random variables defined on (Q, A, P) such that 

(H4) The sequence (O n = i Yh=i $xj»)n>i satisfies an LDP on Mi(S) endowed 
with the weak convergence topology with good rate function I x . 

(H5) For every n > 1 the vectors (X™, . . . , X ™) and (W™, . . . , W") are indepen- 
dent. 

An LDP for 

n 
2=1 

holds as a consequence of Theorem 1.1 and Theorem 2.3 in [17] 

Theorem 1.2. The sequence (V n ) n >i satisfies an LDP on M\(M.+ x S) en- 
dowed with the distance A with good rate function 

J(p) = H(p\ Pl ® p 2 ) + I w { Pl ) + I x {p 2 ). (1.8) 
Again, by contraction, an LDP for 

1 ™ 
n 

i=l 

easily follows from Theorem 1.2 

Corollary 1.2. TTie sequence (L n )n>\ satisfies an LDP on Mi(E) endowed 
with the weak convergence topology with good rate function 



KM = inf J(p) 

p:F(p)=u 

inf { inf { / H(p x \ Pl )p 2 (dx) + I w ( Pl )\ + I x (p 2 ) 
inf {JC(u;p 2 ) + I x (p 2 )}. (1.9) 

P2GMi(E) 

/£ follows that for every v G Mi(E) we /iawe if (y) < I x (v). 
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The latter inequality illustrates the smoothing effect of the random weights 
Wi, . . . , W™ on the distribution of L n . We shall see in most examples that for 
classical choices of ((W ! ")i<i<„)„>i and/or ((-^")i<*<n)™>x there exists at least 
one v £ Mi(E) such that K(v) < I x (v). Nevertheless, we have the following 

Corollary 1.3. A necessary and sufficient condition on ((W^")i<i<n)n>i to 
ensure that for every ((-^")i<i< n )n>i and every v £ Mi(S) we have K(i>) = 
I x (v) is that for every v, f £ Mi(£) 



It follows from Lemma 1.1 that is condition is satisfied e.g. when the sampling 
weights are such that I w (y) = H{v\8\) as for the delete-/i(n) jacknife with 
h(n) = o(n), see Corollary 2.5 below. 

The paper is structured as follows. Section 2 presents several applications 
of our main results. There we discuss efficiency issues for both conditional and 
unconditional LDPs. The rest of the paper is devoted to proofs. In particular 
Section 7 contains the proofs of several sampling weights LDPs. Some of these 
are new to the literature. 

2 Examples of applications 

In this section we investigate the LD properties of (£ n )„>i and (i")n>i for sev- 
eral particular choices of ((W™)i<;< n )n>i and/or ((Xf)i<i< n ) n >i- To specify 
our results we need some more notations. For every A > we shall denote by 
V(X) the Poisson distribution with parameter A and by <2(A) the distribution 
of a random variable Y such that XY is "P(A)-distributed. More generally, for 
every A, 7 > we shall denote by J-"(A,7) the distribution of a random vari- 
able Y such that XY is "P ^-distributed. For every positive integers m and n 
and every n-tuple of non-negative numbers (p™, . . . ,f>") such that 5^j=i Pi = 1 
we shall denote by Mult„(m, (p™, . . . the distribution of (Yi, . . . ,Y n ) the 

numbers of balls found in n urns labeled 1 , . . . , n when m balls are thrown in 
these urns independently, each having probability p" to fall in the urn labeled 
1, probability p% to fall in the urn labeled 2, etc... 

2.1 Efron's bootstrap and "m out of n" bootstrap 

For every m, n > 1 the weights (W", . . . , W") for the "m out of n" bootstrap 
arc defined such that 



Classical Efron's bootstrap corresponds to m = n. It emerges from sampling 
with replacement from the urn containing the observed data. We shall assume 






+00 



otherwise. 



-{W[\ W") ~ Mult„(m, (1/n, . . . , 1/n)). 
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that m = m(n) and that the sequence (A„ = m(n)/n) n >\ satisfies lim^oo A„ = 
A > 0. Quite surprisingly we could not find in the literature a reference for the 
following, even in the simple m(n) = n case 

Theorem 2.1. The sequence (S n = — Y%=i ^w n )n>i obeys an LDP on Mf(R + ) 
endowed with the Wi u distance with good rate function H(-\Q(X)). 

First we consider fixed observations ((xf)i<i< n )n>i such that (1.1) holds. 

Corollary 2.1. The sequence (£ n ) n >i satisfies an LDP on Mi(X) endowed 
with the weak convergence topology with good rate function 

K{v; /i) = \H{v\^jl). 

By properly rescaling we immediately obtain for every A > and every measur- 
able A C Mi(E) that 



- inf H(v\fj,) < liminf — -logP(£" E A) < 
veA° n-i-oo m(n) 

< Umsup — !— logP(£ n g A) < - inf H(v\n). (2.1) 

n^oo m(n) ueA 

Next we investigate the LD properties of (L") n >i without any other assumption 
that (H4-H5). To this end we introduce 

Z = { V GM 1 (E) : L x (n)=Q}. 
It follows from Corollary 1.2 and 2.1 that 

Corollary 2.2. The sequence (£")„>i satisfies an LDP on Mi(£) endowed 
with the weak convergence topology with good rate function K such that 

K(u) = inf ' {AffMO + I X (C)} < A inf 

fEAfi(S) »7£2 

Now we consider some particular cases for ((X™) 1 < i <„)„> 1 . First, we assume 
that for every n > 1 the random variables X", . . . ,X™ are independent and 
identically /i-distributed. Then -J^iLi^P ~^ M a - s - ( see Theorem 11.4.1 in 
[11]) and Corollary 2.1 can be interpreted as a conditional LDP. Hence, any 
"m out of n" bootstrap such that lim„_ ! . 00 m(n)/n — 1 (in particular Efron's 
bootstrap) leads to a conditional LDP that coincides with the original LDP 
in this case. This was first established in [2] for Efron's Bootstrap and in [6] 
in the general case. Actually the X", . . . ,X™ need not be iid but only that 
the associated empirical measures satisfy an LDP with rate function H(-\fi) 
for Efron's bootstrap to be conditionally LD-efficient. Corollary 2.2 completes 
the previous result with an unconditional LDP. In this particular case I x (C) = 
H((\fi) and by taking e.g. jti = ^<5 + y^i, v = ±5 + ±6i, ( = |<5 + 
and A = 1 we observe that K(v) < H(v\Q + H((\n) < H(v\n) hence Efron's 
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bootstrap is not unconditionally LD-efficient. Straightforward use of the same 
kind of arguments shows that this remark also holds true when X", . . . , X™ is 
the result of sampling without replacement from an urn with suitable properties 
(see Theorem 7.2 in [8] for the reference LDP) or when X", . . . ,X% are the n 
first components of an infinitely exchangeable sequence of random variables (see 
[9] for the reference LDP). 



2.2 lid weighted bootstrap 

The weights (W" , . . . , W") for an iid- weighted bootstrap are defined on the 
ground of a sequence Yi, . . . , Y n , . . . of R+- valued independent random variables 
with common distribution £. We shall assume that for every a > we have 
Af(a) < oo and that A|(0) = oo (or equivalently P(Yi = 0) = 0). The weights 
(Wf, . . . , W") are defined by 

V V Y 

w? = — — — w n = — — — w n 



In order to describe the LD behavior of (S n = ^ Ya=i &wv-)n>x we introduce 
the map 

g ■. w 1 (k+) x m; -> W 1 (M+) 

(p, to) h-» g(p,m) : A £ B R+ p(mA) 

where for every Borel set A £ £>r + and every m > we write 

?7i^4 = {x £ R + , 3y £ A : x = my} . 

The continuity of Q is the main argument in the proof of the following 

Theorem 2.2. The sequence (S' a = — Y27=i $w n )n>i satisfies an LDP on 
A/^(R + ) endowed with the Wi,\.\ distance with good rate function 

I w (p)= inf ■ \ H (g(-, P )\o 

m>0 [ TO 

The previous LDP leads to 

Corollary 2.3. For every v, fi £ Mi(S) we have 

Uv a)> { " lf m>0 ^ A « if v <Si p 

' — \ +oo otherwise. 

Moreover, the preceding turns out to be an equality for every v,p £ A/i(E) if 
and only if 

lim A f (a) = and lim A f (a) = +oo. 



It follows from the previous corollary that there is no distribution £ such that 
for every u,pE Mi(S) the identity K.(v; p) = H(v\p) holds. Indeed, as soon as 
there exists v,p € Mi(S) such that = on a set A such that p(A) > one 

has JC(v; p) = oo while it could be possible that if {v\p) < oo. In words there 
is no choice of £ for which one gets a conditional LDP that coincides with the 
original one for X™, . . . , X™ independent and //-distributed. It is clearly due to 
the fact that A|(0) = oo forces all the weights W™, . . . , W™ to be positive which 
is to be compared to e.g. Efron's bootstrap. Finally, as for Efron's bootstrap, 
one can construct examples to show that in most classical cases the iid-bootstrap 
is not unconditionally LD-cfficient. 



2.3 The multivariate hypergeometric bootstrap 

Let A be a fixed integer number such that K > 2. The multivariate hyper- 
geometric bootstrap emerges from the following urn scheme: Put A copies 
of each observed data in an urn so that the urn contains An elements then 
draw from this urn a sample of size n without replacement. The sampling 
weights (W™, . . . , W") take their values in {0, 1, ... , K} under the constraint 
S™=i = n an( i are distributed according to 

P(WT = «;? J ...,W^ = = 



Let us denote by 03 (A, ^) the Binomial distribution with parameters K and 
K' 

Theorem 2.3. The sequence (S n = — X)"=i )n>i satisfies an LDP on 
M 1 1 (R + ) endowed with the Wi,\.\ distance with good rate function 

I w (p) = H(p\<B(K,±)) 

Consider fixed observations ((a^)i<t<n)n>i such that (1.1) holds, we obtain 

Corollary 2.4. The sequence (C n ) n >i satisfies an LDP on Mi(S) endowed 
with the weak convergence topology with good rate function 

+oo otherwise. 

We only obtain an inequality since 53 (if, -^) does not satisfy condition (1.7). 
Again, there is no integer A such that for every u, p G Mi(S) the identity 
/C(z^; /i) = H(v\p) holds. Indeed, as soon as there exists v, p S Mi(S) such that 
> A' on a set A such that p(A) > one has K{v; p) = oo while it could 
be possible that H(y\p) < oo. Thus all multivariate hypergeometric bootstraps 
fail to be conditionally LD-efficients for iid observations. One can construct 
examples to show that in most classical cases the multivariate hypergeometric 
bootstrap fails to be unconditionally LD-cfficient. 
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2.4 A bootstrap generated from deterministic weights 

The weights for bootstrap schemes defined from deterministic weights are given 

by 

(W 1 " ! ...,W^) = « (1) ,...,< n(n) ) 

where for every n > 1 the w™ , . . . , W™ are fixed non- negative real numbers such 
that Yh=i w r = n an< ^ 

1 ™ 

n f— ' 

i—i 

and o~ n is an uniformly over & n distributed random variable. We clearly have 

Theorem 2.4. The sequence (S n = — Yl7=i ^w n )n>i satisfies an LDP on 
7\/ 1 1 (R + ) endowed with the W\ u distance with good rate function 



I W {P) 



if p = 7 
f-oo otherwise. 



An important special case is the grouped, or delete-h jacknifc. The grouped 
jacknifc with group block size h may be viewed as a bootstrap generated by 
permuting the deterministic weights 

«,...,<) = 

\ a — , 
\ 1 

Tl — h 

We shall take h = h(n) such that limn^oo h(n)/n = a G [0, 1) so 




7 = (1 — q)5j_ + a<5. 



o- 



Corollary 2.5. If a > the sequence (£")„>! satisfies an LDP on Mi(E) 
endowed with the weak convergence topology with good rate function 



(1 - a)H{ v \p) + aH G M^E) 

+oo otherwise. 



If a = i/ie sequence (£")„>i satisfies an LDP on Mi(E) endowed with the 
weak convergence topology with good rate function 

I +oo otherwise. 

Naturally this result coincides with Theorem 7.2.1 in [8]. Combining Corollary 
1.2 and Corollary 2.5 we obtain an unconditional version of the latter result. To 
every v 6 A/i(E) we associate 

£, = {CGMi(E) : C-(l-Q> gMl (S)} 
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Corollary 2.6. If a > the sequence (L™) n >i satisfies an LDP on Afi(E) 
endowed with the weak convergence topology with good rate function K{v) = 
inf^gg^ U(v, Q where 

U{v, - (1 - a)H{v\Q + aH ( C ~ ^~ a)v \A + I X (C). 

If a = the sequence {L n ) n >i satisfies an LDP on Mi(S) endowed with the 
weak convergence topology with good rate function K{v) = I x (y). 

2.5 The A>blocks bootstraps 

Let us consider the (moving or circular) fc-block bootstrap. Consider weights 
from the "m = n/k out of n" bootstrap: 

. . . , W£) ~ Mult„(m, (l/7i, . . . , 1/n)). 

The fc-blocks bootstrapped empirical measure can be written as in [20] with the 
formula 

where j ~ i means that the j belong to block i. For the moving fc-blocks 
bootstrap, j ~ i if j G {i — fc/2, • • ■ , i + miodulo n. With slight modi- 
fications we could also consider the circular fc-blocks bootstrap where j <~ i if 
j G {i, . . . , i + k — 1} modulo n. Both schemes are asymptotically equivalent as 
soon as fc is fixed as it is the case here. Notice that 

i ™ i 

where (W", . . . , W£) fails to be exchangeable. However, our approach relies 
on preliminary results like Theorem 1.1 that are general enough to allow us to 
handle this situation under some mild additional hypothesis. Indeed, assume 
that the observations ((xf)i<i< n ) n >i satisfy 

i=l 

the i indices being taken modulo n. This situation arises e.g. when we are given 
the realization x™, . . . , x™ of X™, . . . , X™ the first n components of a stationary 
Markov chains (Yi)i>i with transition probability P and stationary measure pi 
as in [10] (see also [5]). In this case we get 

1 " 

- Y J 6(x?,...,x^_,) 4^ f^-£P a.s. 
1=1 fc-i 
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so (2.2) is satisfied with fj,^ = [i (g> P (g> ■ • • g> P . which leads to the following 

fc-i 

conditional LDP 

Theorem 2.5. The sequence (C n ) satisfies an LDP on M±(S) endowed the 
weak convergence topology with good rate function 

K{r t »M) = inf {i^W^*)) : „« £ Ml (Z k ) , ^ ^ = "} 

i=l 

In the particular case of iid observations we further obtain 

Corollary 2.7. The sequence (C n ) satisfies an LDP on Mi(£) endowed the 
weak convergence topology with good rate function 

K.(v-^ k) ) =H(v\n). 

hence the fc-blocks bootstrap is conditionnally efficient in this case but fails to 
be unconditionally efficient for the same reason as Efron's bootstrap, see Section 
2.1. 



3 Proof of Theorem 1.1 

Most of the proof of Theorem 1.1 relies on the proof of a particular case 
that we describe now: We are given two triangular arrays ((w™)i<i<„)„>i and 
((x™)i<j< Tl ) n >i of elements of K+ and S respectively, possibly with repetition, 
such that 



v i.n = ±J2 S K ^ v 1 e and v 2 * n = - ^ 5 xf 4 v 2 6 Mi(E), 

»=i n »=i 

and such that for every n > 1 we have w i = n - Let (<T n ) n > 1 be a se- 

quence of random variables defined on (fl,A,F) such that for every n > 1 the 
distribution of o~ n is uniform over 6 n . Let (T n )„>i be the sequence of random 
measures defined on (fl,A,f) by 

1 " 

t=l 

This is a particular case of V™. Following the proof of Theorem 1 in [25] we 
shall prove 

Lemma 3.1. The sequence (T n )„>i satisfies an LDP on Mi(M.+ x S) endowed 
with the weak convergence topology with good rate function 

ji \ _ [ P^ipW 1 <8 «/ Pi = v 1 and p2 = v 2 

\ +oo otherwise. 
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A stronger version of Lemma 3.1 i.e. an LDP for (T")„>i on _M}(R+ x £) 
endowed with the distance A is proved in Section 3.2. The final proof of Theorem 
1 relies on the latter result and Theorem 2.3 in [17] and is given in Section 3.3. 



3.1 Proof of Lemma 3.1 

Let us introduce some more notations before we carry on with the proof of 
Lemma 3.1. For every n > 1 we write 

V n = jf G M{(R+ x £) : 3 a G 6„, v= ^^« w ,^)| ■ 

We introduce two triangular arrays ((£™)i<i<n)ra>i and ((i?™) l<z<n)n>l 

of el- 
ements of R + and E respectively, defined on (Q, A, P) and such that for every 
n > 1 the 2n random variables L", . . . , L™, i?™, . . . , i?" are mutually indepen- 
dent. We further assume that every L™ (resp. R™) is distributed according to 

v l,n ( reg p j, 2 ,n^ ip^g SG q uence 

r = -ff(i ?lB » ) £Mi(K + xS) 

n Z — / > 

i=l 

has the following LD behavior 

Lemma 3.2. TTie sequence {T n ) n >i satisfies an LDP on Mi(R + x E) endowed 
with the weak convergence topology with good rate function H{p\v 1 ® v 2 ). 

Proof Since v l ' n ^ v 1 we have i/ 1 '" A z^ 1 according to e.g. Theorem 7.11 in 
[26]. Moreover, since v 2 ' n A v 2 we have v l n ® v 2 n —> v 1 ® v 2 (see [3], Chapter 
1, Theorem 3.2). The announced result then follows from Theorem 3 in [2]. □ 

Our strategy in proving Lemma 3.1 consists in comparing T n to random mea- 
sures coupled to T™. Comparison is possible because the p £ Mi(R + x E) 
such that I(p) < +oo can be approached in the weak convergence topology by 
elements of V n - Our proof of this property requires to use several metrics on 
A/i(R+ x E) compatible with the weak convergence topology. This is the reason 
why in Section 3.1.1 we give a short account on the weak convergence topology 
prior to the proof of our approximation result. In Section 3.1.2 we construct 
our coupling. Finally, in Section 3.1.3 we prove the LD bounds of Lemma 3.1. 

3.1.1 An approximation result 

We are given the Polish space (R+,<i e ) with d e (x,y) = \x — y\ . The distance d e 
is not a bounded metric but it is topologically equivalent to the bounded metric 

1 We use this notation to underscore the fact that for Lemma 3.1 to hold it is not necessary 
for the to™'s to be real numbers. It is sufficient to have Polish space valued fu^'s. 



14 



de{x,y) 



d e (x,y) 
1 + d e {x,y) 



Analogously we define a bounded metric c?s on E on the ground of g?£. The 
product topology on R + x E is metrizable by e.g. 



d2,M((xi,x 2 ), (2/1,2/2)) = max(d e (a;i, j/i), (i E (^2,2/2)) 



(3.1) 



or 



d-2,+ {{xi,x 2 ), (2/1,2/2)) = 4(2=1,2/1) + d s (x 2 ,y 2 ). (3.2) 

They both make R+ x £ a Polish space. We can also metrize the product 
topology on R + x £ with the analogues c?2,m and d 2 ,+ of (3.1) and (3.2) built 
on the ground of d e and d-£. With a slight abuse of notation we shall denote by 



sup 

ll/lloo + ll/llrj: 



fdp 



fdv 



(3.3) 



the so-called dual-bounded Lipschitz metric on A/i(R + ) where 5 is either d e 
or d e . It is compatible with the weak convergence topology (see [11], Chapter 
11, Theorem 11.3.3). According to Kantorovitch-Rubinstcin Theorem (see [11], 
Chapter 11, Theorem 11.8.2) the following metric on Mi(R+) 



inf 

?eC( Pll /) 



d e (x,y)Q(dx,dy) 



the so-called Wasserstein-1 metric associated to d e is compatible with the weak 
convergence topology as well. However, note that the "analogue" of W 1 g built 
on the ground of d e is not a metric for the weak convergence topology (for an 
illustration of this fact see [11] p. 420-421). Finally we shall denote by 



BL., 



sup 

f£C b (M + xS) 
ll/lloo+ll/llr j 



fdp 



.xi 



fdv 



(3.4) 



and 



W i d 2+ (P^) = „ i nf , W d 2 ,+ {x,y)Q(dx,dy) \ , 

QGC(pM) [J(R + xS)x(E+xS) J 

two metrics on Mi(R+ x E) compatible with the weak convergence topology on 
this set. The following is a key result in the proof of Lemma 3.1. 

Lemma 3.3. Let p £ Mi(R + x E) be such that p\ = v 1 and p 2 = v 2 . For every 
n > 1 there exists a p n € V n such that p n —> p- 
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Proof Let p 6 Mi(R + x S) be such that p\ = v 1 and P2 = v 2 . According 
to Varadarajan's Lemma (see [11], Chapter 11, Theorem 11.4.1) there exists a 
family ((«", u- 1 )i<i< n )n>i 01 elements of R + x £ such that 



n 

7 = - >.<W,t>?) ->■ P- 

i=l 

For every n > 1 we take <^„, r„ G & n such that 

n f n 

and 



i=l L i=l 



We shall prove that 

1 " 

n \ T c- w 

p = -Vv ...) -> p- 

i=l 

To this end it is sufficient to prove that (3 BL j 2 m (p n , p) — > 0. Let e > be fixed. 
There exists an Nq such that for every n > No we have 

pBLA, M (p,7 n )<e/Z- (3-5) 

Since 7" — > pi = u 1 there exists an N\ such that for every n > Ni we have 
r (7™, i/ 1 '") < e/3. We will show that due to this for every n > Ni we have 

n 

-^Je«,< nW )<£/3. (3.6) 

i=l 

We shall prove that it leads to 

P B LJ 2 ,jr,l n )<e/3 (3.7) 

for every n > Ni where 

1 " 

Y n = - / ^(w n ,.,,«?)• 
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Analogously to (3.6 - 3.7) one can prove that there exists an N2 such that for 
every n > N2 

fe lA ,„(7",/)<E/3. (3.8) 

By combining (3.5, 3.7, 3.8) we obtain the announced result. 

Proof of (3.6) Since 7" and v have finite support every Borel probability 
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measure Q on 



such that Qi = 7™ and Q-j 



is of the form 



1 " 



(3.9) 



where a = {o.i,j)i<i t j< n is an n x n bi-stochastic matrix. Conversely every n x n 
bi-stochastic matrix a defines through (3.9) a Borel probability measure Q{a) 



such that Q(a)i — 7™ and Q(a) 2 



,l,n 



From the Birkhoff-Von 



on K_|_ x 

Neumann Theorem we know that every bi-stochastic matrix can be written as 
a convex combination of permutation matrices. These are n x n matrices with a 
single 1 in every line and every column, all other entries being (for a proof of 
this fact see e.g. [22], Chapter 11, Example 11.2). There is an obvious one-to- 
one correspondence between elements of 6 n and n x n permutation matrices. 
For every S S & n we shall denote by Kg the permutation matrix naturally 
associated to S by this correspondence. Therefore for any Borel probability 



measure Q on 



1+ such that Q\ = 7™ and Q2 



i.e. any choice of the 



components [\s)see rl of the convex combination a ~ J2se6 n ^sKg such that 
Q = Q(a) we have 



d e (x,y)Q{dx,dy) 



> 



see„ 1=1 

n 

-£*.(<,«&<<>)• 



(3.10) 
(3.11) 



Hence for every n > N\ 



1 n 

< e/3 



which proves (3.6). 

Proof of (3.7) For every / e C b (R+ x E) such that ||/||oo + \\f\\ L ,i 2M < 1 we 



have 



/ fdT - I 



fdrf 



xE 



1 I 

i=i 

n 

< -E^a/(k„ (4) , <),«,<)) 

1=1 

1 " 



< 

n 

i=l 

< e/3. 
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Hence for every n > N\ we have (3 BL j 2 (7™, 7™) < e/3. □ 
3.1.2 Coupled empirical measures 

To every n > 1 and every realization of T" we associate two elements of Mi (R + x 
S)by 

W U2 jr\Wn = mm + CT»} (3.12) 

and 

WW^N = ™% {w iA JT n ,^} . (3.13) 

In case there are several elements of V n achieving the min (resp. the max) W n 
(resp. W n ) is picked uniformly at random among these measures. 

Lemma 3.4. For every n > 1 the random measures W n , W n and T n are iden- 
tically distributed over Mi(K + x E). 

Proof We shall only prove that W n and T n are identically distributed since the 
proof with W n and T n works the same way. Let n > 1 be fixed. For the sake of 
clarity let us first assume that there is no repetition among the u>™ , . . . , w™ and 
the x", . . . , x™. In this case every p £ "P n corresponds to a single r G & n by 

1 " 

i=i 

Next let us consider a fixed realization (If, rf)i<i< n of (L™, i?")i<i< n and let 
us denote by w n the corresponding value for T™. Due to the property of the 
minimizcr in the Wasserstein distance between two atomic measures we already 
employed in the proof of Lemma 3.3, for every p 6 V n (i.e. every r 6 & n 
according to (3.14)) there exists a a € & n such that 

W h j 2 Jw n ,p) = iX)4+((i?.'T),« w> x? 0T(i) )) 

1 n 1 n 

= - E + - E ^ r ".c ( „). 

i=l i=l 

Thus, since we are looking for the minimum over a and r, for a fixed realization 
(Z",r")i<i<„ of (L™, i?")i<j<„ the corresponding value of w™ is obtained by 
finding 77; and 77,. such that 

n f n 

E«,< (i) ) = ™ E«><«) 

i=l " I i=l ) 
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and 



and taking 



i=l 




1 ™ 

n * — ' u( 



n 

i=l 

In case several r/i and/or r\ r realize the minima in the displays above, those 
defining w™ are picked among them uniformly at random. Now, remark that 
for every 7^7,. £ & n , observing (i™^),^^ )].<*<„ has the same probability as 
observing (Z™, r")i<i<„ and results in 7; or;; and ~f r or) r in defining w™ instead of 
j]i and r/ r . Thus, if we consider rji and ry r as random variables defining W n ,jwe 
see that their distribution conditioned on W n is uniform over ©„. Hence W n 
and T n are both uniformly distributed over V n , thus identically distributed. 
This proof extends easily to the case when there are repetitions among the 
w?,...,w% or xy,...,a£. □ 

3.1.3 Proof of the LD bounds of Lemma 1 

We start the proof of the LD bounds by proving the following 
Lemma 3.5. We have: 

1. I is a good rate function. 

2. The sequence (T n ) n >i is exponentially tight. 

Proof 

(1) Let a > 0. We have 



N a = {p e Mi (R + x E) : I{p) < a) 

= {pe Mi(R+ x S) : H{p\v l <g> v 2 ) < a} n {p € M X (R + x S) : pi = i/ 1 and 

Thus, for every a > 0, iV Q is the intersection of a compact and a closed subset 
of Mi(R+ x £), therefore it is compact. 

(2) For every measurable A C Mi(R + x S) we have 
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1 1 1 " 1 n 

limsup - logP(T" G ^4 C ) = limsup - logP(T" G A c |- V <5i« = v l > n , - V<5j 



R n = V 2 ^ 

n n z — ' 1 n t — ' 

i=i »=i 



< limsup - log P(T" G A c 

n— >-oo W 



n 

liminf-logP(- V(5 L ™ = z^'") 
n-i-oo n n — ' 1 

i=l 

1 1 - 

liminf - logP(- V = v 2 - n ). 



n ' 

1=1 



Since (T n )n>i satisfies an LDP on Mi(R+ x S) with a good rate function it is 
exponentially tight (see [8], Remark a) p. 8). Thus for every a > we can chose 
a compact set A a C Mi(R + x E) that makes the first term in the last display 
smaller than —a — 2. On the other hand it is clear that for every n > 1 we have 

n 1 
P(-Vtfi,» = i/' n ) > n! — 

2=1 

equality corresponding to the case when there are no ties among the x™, . . . , x™. 
Thus 



1 1 - 

-liminf-logP(- > 5i» = i/ 1,n ) < 1 

i=l 

which completes the proof. □ 



Proof of the lower bound 

It is sufficient in order to prove the lower bound of the LDP to prove that 

-I{p)< liminf - logP(T™ G B(p,e)) 

holds for every p G Mi(R + x S) and every e > 0, where B(p,e) stands for the 
open ball centered at p G Mi(R+ x £) of radius e > for the W 1 j 2 metric. 
So let e > and p G Mi(R+ x S) be such that /(/>) < +oo. In particular 
pi = v 1 and p2 = v 2 . According to Lemma 3.3 there exists a sequence (p n ) n >i 
of elements of Afi(R + x S) such that p" G V n and p" — > p. According to Lemma 
3.4 



\T n G B(p, e)) = P(VF" G B(p, e)) 

> 7*) < |,W lA , + (T T, ,P n ) < |,Wi >4li+ (p n ,p) < |) 
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since it follows from the definition of W n that for every p" G V n we have 

W lA ^\T l )<W l42 + ( P \T n )- 

On the other hand since p" p we get that for n large enough j 2 + (p™, p) < || 
f2. Thus, for those n's 



> nw hd2 jr\ P )<-). 



Finally, it follows from Lemma 3.1 that 



liminf - logP(T" 6 B(p, e)) > liminf - logP(^ 1 ^ (p, T") < |) 

> -//(p^ 1 ® = -J(i/). 

Proof of the upper bound 

In order to prove the upper bound of the LDP, it is sufficient to prove that it 
holds for compact subsets of Mi(R+ x E). Indeed, since (T")„>i is an expo- 
nentially tight sequence (see Lemma 3.5) the full upper bound will follow from 
Lemma 1.2.18 in [8]. Let A be a compact subset of Mi(R+ x E) and let us 
denote by 

A u iy-i = {p G A : pi = v 1 and p2 = v 2 } 

which is a compact subset of Mi(R+ x E) as well. Since the weak conver- 
gence topology on M±(M.+ x E) is compatible with the j 2 + metric, it makes 
Mi(R+ x E) a regular topological space: For every p G A such that p G 
A c vl v 2 there exists e p > such that B(p 7 2e p ) D A u iy = 0. In particu- 
lar B(p,e p ) n A v iyi = where B(p,e) denotes the closed ball centered on 
p G Mi(R+ x E) of radius e > for the W 1 j 9 + metric. On the other hand, 
since p t— > H(p\v 1 ®v 2 ) is lower semi-continuous, for every p G A u iyi and every 
5 > there exists a <p(p, (5) > such that 

inf H^v 1 ®!^ 2 ) > (H(p\^ (g)iy 2 ) - S) a\. 
For every S > we consider the coverage 

Ac (u peAnAu ^B{p,e p )^ U fupeA^^Bip,^^-)^ 
from which we extract a finite coverage 

A c (U peh B(p,e p )) U (u pel2 B(p, ^%^) 
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where 1\ C A n v2 and 72 C A v \ v i are finite sets. Then, according to 
Lemma 1.2.15 in [81 



lim sup - log P(T" G A) < max I lim sup - log P(T" e U peIl B{p 7 e p ) n A), 

hmsup i logP(T" G U pel2 B(P, ^^)) 
For every p G I\ there can not be an infinite number of integers such that 

¥{T n » eu peIl B( P ,e p )nA)^o 

for otherwise we would get B(p,e p ) PI A„i v i ^ 0. The first term in the max is 
then equal to — oo. We are left with the second term and according to Lemmas 
1,2 and 3 we have 



lim sup - log P(T" e A) < lim sup - log P(T n e U pel2 B< 



V, 



)) 



< max 

peh 

< max 

< max 
peh 

< max 
pe/ 2 

< max 

< max 
peh 
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)) 



limsup-logP(T^ n e B(p. 

lim sup 1 log P(Wj j 2 + W 7 ' 1 ) < 

hmsup I log P(W lA+ (p,W n ) < 

inf {~f\v l ® i/ 2 ) 

ieB(p,<p{p,&)) 

o 



<I(P)-6)A- 



< - inf ((/(,) -*) A- 

By letting (5 — !> we obtain the announced upper bound, see Remark 1.2.10 in 
[8]. 

3.2 A stronger version of Lemma 3.1 

In the present section we shall prove the following 

Lemma 3.6. The sequence (T n )n>i satisfies an LDP on A4\(R + x E) endowed 
with the distance A with good rate function I . 
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Proof First let us notice that 

A/' 1 1 (M + x E) = jp G M!(M + xE) : J x pi{dx) < l| 

is a closed subset of Mi(R+ x S) when the latter is endowed with the weak 
convergence topology. Since for every n > 1 we have P(T„ G A/^ 1 X E)) = 1, 
Lemma 4.1.5 in [8] implies that (T„) n >i obeys an LDP on A/i(M+ x E) endowed 
with the weak convergence topology, with good rate function /. 

Next we prove that the same remains true when 7V 1 1 (R+ x E) is endowed with 
the distance A. Indeed, since (T")„>i satisfies an LDP on the Polish space 
M 1 (R + x E) with a good rate function it is exponentially tight: For every L > 
there exists a compact A C M 1 (R+ x E) such that 

limsup - logP(T" G A c ) < -L. 

n—too 

The set A n K with 

K = {p G M\R + x E) : pi G U? =1 {*/' n }} 
is a compact subset of A/" 1 1 (M+ x E) endowed with A. Moreover 



lim sup - log P(T n 6 (in K) c ) < lim sup - log P(T™ G A c ) + 

n— >oo Tl n—>oo 

+ lim sup - log P(T n G K c ) 
n—>oo ti 

< -L. 



Finally, M\(R+ x E) is a closed subset of (jV 1 1 (R+ x E), A) and for every n > 1 
we have P(T„ G .M}(M+ x E)) = 1 so again, due to Lemma 4.1.5 in [8] the 
sequence (T„) n >i obeys an LDP on .M}(M+ x E) endowed with the distance A 
with good rate function /. □ 

3.3 Conclusion of the proof of Theorem 1 

In order to conclude the proof of Theorem 1 it is sufficient to establish that the 
distribution of V™ on A^J(M+ x E) is a mixture of Large Deviation Systems 
(LDS) in the sense of [17]. For the sake of clarity we recover the notations of 
[17] when identifying the components of the LDS 

• Z = A^i(R+ x E) is a Polish space when endowed with the distance A, see 
the Appendix. 

• X = Mi (R+) is a Polish space when endowed with the Wi-distance since it 
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is a closed subset of the Polish space (W 1 (R+), Wi ; |.|) (see e.g. [4]). 
• For every n > 1 we note 



X n = \v G A/ 1 1 (M+) : 3 (toi, ...,«;„) G (R+) n , v 



1 E^1 

i=l J 



and for every v G A? and every n > 1 there exists a c" £ X„ such that v n ™-> v, 
see Lemma A.l in the Appendix below. 

• The map 7r : Z — > X defined by tt(v) = v\ is continuous and surjective. 

• For every n > 1 and every i/ = X)"=i e ^ n ^ ^ e the distribution 

of T n = i EILi <W W .*?) under The famil y n = { p "> ^ 6 *n, ™ > 1} Of 

finite measures on the Borel er-field on Z is such that for every n > 1 and every 



G we have PJ'C 71 " (M c ) = 0. 



• Let Q n be the distribution of ^ Y17=i $wp- For every n > 1 and every and 
every measurable A C A^}(R+ x S) 

P(V" e A) = ( Pl l {A)Q n {du). 
Jx„ 

All the requirements of Definition 2.1 in [17] are satisfied by our model thanks 
to Lemma 3.6. It follows from Theorem 2.3 in [17] that the sequence (V")„>i 
obeys an LDP on A4j(M+ x X) with distance A with good rate function 



J(p) = 



H(p\p 1 ®p)+I w {p l ) if p 2 = p 
+oo otherwise. 



4 Proof of Lemma 1.1 

In order to prove (1.6) it is sufficient to consider is, p G A/i(S) such that 
K,{v] p) < oo for otherwise the inequality trivially holds. Then there neces- 
sary exists (at least) ape A / Jj(IR + x S) such that P2 = p and F(p) = v. For 
every such p the latter reads 



v{A) = / wp x (dw)p(dx) 

JR + xA 

for every measurable A C X, hence v <C p and 

wp x (dw) (4.1) 



p a.s. This shows that v <C p is a necessary condition for the existence of p such 
that pi = p and .F(p) = f so, as claimed, the announced inequality holds true 
as an equality if v <C p is not satisfied. Moreover, we have 

H(p\ Pl ®p) + H{ Pl \t) = H(p\£®p) 

H(p x \0p(dx) 
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since P2 = p. It follows from Kullback's inequality (see e.g. Theorem 2.1 in 
[21]) that 



H(p x \0 > A|( / w Px (dw)) 



which combined with (4.1) brings the announced inequality. Actually it shows 
that a necessary and sufficient condition on £ to get for every Mi(£) an 

equality in (1.6) is that for every Mi(£) there exists ape A^J(M+ x £) 

such that p2 = p, F(p) = v and 

dv 



n{ Px \i) = k\{—{x)) 



p a.s.. According to Kullback's inequality this holds true if and only if there 
exists fi x G R such that p a.s. p x can take the form 



Px(dw) 



1 

— — ( 

Z x 



adw) 



(4.2) 



and still satisfy 



wp x {dw) =^-{x). 
dp 



Let us recall that under (1.4) the map A^ is C°° in R, that for every a G 

L we^(dw) 



A J a) 



Is 



°£(dw) 



and that A^ (a) > for every a G R, see Section 2.2.1 in 
that 



So finally it appears 



lim At (a) = and lim At (a) = +oo 

a— >+oo 



is a necessary and sufficient condition to get for every v,p G M\ (£) and every 
x G S such that ^(a:) >0ap x of the form (4.2) with / K wp x (dw) = §£(x) 

and if(/9 x |£) = If = then one takes = <$o and still gets 

/ R wp x {dw) = ^(x) andi?(p K |£) = A*(^(x)) = A*(0). The announced result 
follows. 



5 Proof of Theorem 1.2 

In order to prove Theorem 1.2 it is sufficient to establish that the distribution 
of V n on 7Wj(R+ x S) is a mixture of LDS. Again we recover the notations of 
[17] when identifying the components of the LDS 

• Z = Ai\(M. + x S) is a Polish space when endowed with the distance A. 

• X = Mi(£) is a Polish space when endowed with the weak convergence 
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topology. 

• For every n > 1 we note 

X n = ji/ e AfftE) : 3 (x!,...,x n ) e E n , = ^X]^} 

and according to Varadarajan's Lemma for every v £ X and every n > 1 there 
exists a v n £ X n such that v n — > f. 

• The map 7r : Z — > X defined by tt(v) = v 2 is continuous and surjective. 

• For every n > 1 and every v = — J27=i € %n let -P™ be the distribution 
of T n = i ^=1 <Vf ,x ( under P. The family 'il = {P™, ^ G <Y„, n > 1} of finite 
measures on the Borel er-ficld on Z is such that for every n > 1 and every v £ X n 
we have {-k^ 1 ({v} c ) = 0. 

• Let Q n be the distribution of — X)"=i ■ For every n > 1 and every and 
every measurable A c x E) 

P(V" £ A) = (' P?{A)Q n (dv). 

All the requirements of Definition 2.1 in [17] are satisfied by our model thanks 
to Theorem 1.1. It follows from Theorem 2.3 in [17] that the sequence {V n ) n >\ 
obeys an LDP on _M];(R+ x E) with distance A with good rate function 

J(p) = H(p\ Pl ® p 2 ) + I w { Pl ) + I X { P2 ). 



6 Proof of Corollary 1.3 

In view of (1.9) a necessary condition on ((M / ")i<i< n ) n >i for K = I x is that 
for every v, ( £ M X (E) and every ((Jff )i<i<„)„>i 

I x {u)-I x (Q<lC(y,£). 

If we consider X™,...,X™ resulting from sampling without replacement on 
an urn which composition x™, . . . , x ™ satisfies i $et> ^> C we know that 

— &rr* satisfies an LDP with good rate function 



I x (9) 



if 9 = ( 
oo otherwise. 



so the announced condition is necessary. It is also clearly sufficient and the 
announced result follows. 



7 Sample weights Large Deviations Principles 

In this section we prove the results given in Section 2. 
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7.1 Efron's bootstrap and "m out of n" bootstrap 
7.1.1 Proof of Theorem 2.1 

The proof relies on the combination of a coupling construction and a Sanov's 
result in a strong topology. 

Coupling Poisson and Multinomial distributions. 

Let m, n > 1 and Z" , . . . , Z™ be independent random variables such that every 
Z™ is P(m/ri)-distributcd. We shall interpret each Z" as the number of balls 
randomly thrown in an urn labeled i. To (Z™, . . . , Z™) we couple (M™, . . . , M") 
in the following way 

• ^ Sr=i = 171 we * a ke -^T = f° r ever y 1 < « < n. 

• H Sj=i -> m we pick uniformly at random X)"=i ^i" — TO Da h s in the urns. 
We define (M™, . . . , M") as the new occupation numbers of the urns. 

• If Y17=i < mwe add m — Ym=i balls into the urns. The urns are chosen 
independently for each added ball with probability 1/n. Again, we denote by 
(Mi, . . . , M") the new occupation numbers of the urns. 

Lemma 7.1. The vector (Mf , . . . , M") is Mult n (m, (1/n, . . . , 1/n)) -distributed. 
The coupling is optimal in the following sense: 

Lemma 7.2. For every non-negative real numbers x\, . . . ,x n such that XL =1 X{ = 
n we have 

1 n i n 1 n i n 

^M(-E%M r ,-E%^r)<^i-i(-E^'-E%^)- 

i—1 i—1 i—1 i—1 

Proof of Lemma 7.1 Remark that the conditional law of (Zi, . . . , Z n ) given 
S"=i = k is Mult„(fc, (1/n, . . . , 1/n)). Thus, if k = m the result is obvious. 
If k < m or k > m, the coupling is done such that we move from the law of 
k balls randomly putted in n urns to m balls into the same urns by picking or 
putting at random. We finally get the Mult„(m, (1/n, ... , 1/n)) law. □ 

Proof of Lemma 7.2 By definition 



-.71 I n fl 71 1 n 

wi, M (-E%^^E%^) = su p 1 E/(-^r)- 1 E/(-^ n 

n * — ' m 1 n * — ' m * /EOiinj.) n ^ — ' to n * — ' m 



1 " 

^ -Ei M "-^i 



Due to the construction of (Mf, . . . , M") we have that either all M™ - Zf < 
or all Mf - Zf > 0, so 
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1 " 



z?\ = 



n 

1=1 

1 " 

TO ' 



Moreover, for every non-negative real numbers X\ 
we have 



, . . . , o> n 



such that Yli=i x i 



n'f-f ra-f-f m * /ec l( R + ) \ n , m 

1=1 1=1 Zri V l =l l =l / 



> 



which concludes the proof. 



- -2?) 
m 



1 
i 

1 - 1 E^ 

TO 1 



□ 



A Sanov's result in a strong topology. 

Let ((Z™, . . . , Z™)x<i<n)ri>i be a triangular array of random variables such 
that for every n > 1 the are independent and identically V(X n )- 

distributed and such that linin^oo A rl = A > 0. 

Lemma 7.3. The sequence (lZ n = ^ Y^i=i °~-i-z n )ri>i obeys an LDP on W 1 (1R+) 
endowed with the distance with good rate function H(-\Q(X)). 

Proof of Lemma 7.3 According to Theorem 3 in [2] the sequence (lZ n ) n >i 
obeys an LDP on M\ (K+) when the latter is endowed with the weak convergence 
topology with good rate function H(-\Q(Xj). Moreover, for every n > 1 we have 
1Z" G W 1 (K+) and, since all the exponential moments of Q(A) are finite, for 
every v G Mi(R+) if H(u\Q(X) < oo then necessarily v G W 1 (1R+), see e.g. 
Lemma 2.1 in [14]. Thus {K n ) n >i obeys an LDP on W 1 (R+) endowed with the 
weak convergence topology with good rate function H(-\Q(X)), see Lemma 4.1.5 
in [8]. In particular it is exponentially tight i.e. for every L > there exists a 
compact K L C W 1 (M+) such that 

limsup - logP(7e" <£K L )<-L. 

n— >oo Tl 

To strengthen the topology for which the LDP holds it is sufficient to prove that 
the sequence (TZ n ) n >i is exponentially tight in the stronger topology, see 
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Corollary 4.2.6 in [8]. To this end we follow the proof of Theorem 1.1. in [27]. 
For every L > we introduce 



-4, 



jz; S W X (R+) : jf A* Q{x) (x)u(dx) <L 



We prove that for every L > the set Cl = Kl H Al is Wi t \.\ -compact. First 
notice that Al is closed for the weak convergence topology hence Cl is compact 
for this topology. Thus, to prove that Cl is W± \.\ -compact it remains to prove 
that Cl has uniformly integrable first moments. Let S(N) = A*q^(N)/N. 
According to Lemma 2.2.20 in [8] we have lhxiAr^oo S(N) = oo. For every 
v e Cl we have 

1 r A* Q(x) (x) 
xv{dx) < / — xv(dx) 



< 



>N >x>N x 

L 



< 



S(N) 

hence Cl is Wij.i -compact. Since 

limsup - logP(7e" £ C L ) < max (-L, limsup - logP(ft n <£ A L )j (7.1) 

we are led to consider P(7?" Al). According to Chebichev's inequality for 
every 5 <E (0, 1) 

n 

P(ll n ?AL) = P(-^A* Q(A) (Zf)>L) 

i=l 

< exp(-6nL)(Eexp(6A* Q(x) (Z?))y . (7.2) 

But for every x £ R and every A > we have Aq^(x) = A^^(Xx) hence 

^* / \ _ f A - Xx + Xx log a; if x > , , 

2( A )^ \ oo otherwise. ' 

So we get for every i£t the relationship A* Q ^(x) = (A/ X n )A* Q( ^ x ^(x) so 

P(ft" Ai) < exp(-<5nL) (Eexp((5(A/A„)A* Q(A) (zn))" . 
According to Lemma 5.1.14 in [8] for n large enough we have 

Eexp(5(A/A„)A* s(A) (Zn) < 1 _ 
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which combined with (7.1) and (7.2) shows that (TZ n ) n >i is exponentially tight 
in the Wi,\.\ topology. The announced result follows. □ 

Conclusion of the proof of Lemma 2.1 

LD upper bound Let F be a Wi t \.\ -closed subset of M 1 1 (R+). Since M 1 1 (M + ) 
is a closed subset of W 1 (M+), F is also a closed subset of W 1 (K+) . For ev- 
ery n > 1 let Zf,...,Z™ be independent and identically V(X n ) distributed 
random variables and (M™,...,M™) the vector associated to Z™,...,Z™ by 
the coupling construction described in Section 7.1.1. According to Lemma 
7.1 (Ml 1 , M") is Mult n (m(n), (1/n, . . . , l/n))-distributed, which is also the 
distribution of conditioned on {^27=1%™ = m(n)}. According to 

Lemma 7.3 




< -infHHQ(A)) 
since liminf, woo | logP(X)?=i ^" = m(ra)) = 0. 

L£> lower bound Let O be an open subset of Ml (R+). In order to prove the LD 
lower bound it is sufficient to prove that for every pgOwe have 

1 1 - 

liminf-logP(- S^6 W n £ O) > -H(p\Q(X)), 
n-toc n n z — ' ' 

i=i 

So let p £ O and e > be small enough to ensure that B(p,e) C O. We can 
assume that H(v\Q(X)) < oo for otherwise the LD lower bound trivially holds. 
According to Lemma A.l proved in the Appendix for every p £ (R+) there 
exists a triangular array of non-negative real numbers ((xf )i<i<n)n>i such that 

p n = i X)"=i z 9 anc ^ sucn tnat f° r ever Y ^ > 1 we have Xa=i = n - 

Again, let Z", . . . , Z™ be independent and identically V(\ n ) distributed random 
variables and (M™, . . . , M") the vector associated to Z™, . . . , Z™ by the coupling- 
construction described above. In particular 

i—1 i—1 i—1 i—1 
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So, 

2=1 2=1 

1 n 1 n 

> p(Wi,h(-E^^'-E^) <£ A 

i=l i=l 
n 

^i,H(/>": - E 5 ^?) < £ A Wi,|.|(p n ./>) < e/3) 

i=l 

n 

> P(Wi,|.|(p n J -E*^)< e A ^ liM (p™,p)< e /3) 

i=l 
1 " 

> P(^|.|( A -^5 fzn )<£/6) 

'II 77 ^ ^ An 1 



71 ' 
»=1 



for every n large enough and the LD lower bound follows from Lemma 7.3. □ 

7.1.2 Proof of Corollary 2.1 

First notice that Ag( A )(a) = A(e Q / A — 1) so condition (1.7) is satisfied. Using 
(7.3) we immediately obtain the announced result. □ 

□ 

7.1.3 Proof of Corollary 2.2 

Let us prove the inequality in Corollary 2.2. Let v G Mi(£) be such that 
ini^z H(v\rj) < oo. Necessarily for every 77 £ Z such that H{v\rj) < 00 we 
have v <C r\- Now for every such 77 consider p G .A/f } (K.+ x £) defined by p-2 = f] 
while the regular conditional distribution of its first marginal given the second 
is 

F(\,\^(x))(dw) if£(z)>0 



pJdw) = < ; . dr > . 

^ do(dw) otherwise. 

Let us check that F(p) — v. Indeed, every measurable A C S can be decomposed 
into A = A y U A„ where A v = A n Support (z/) and 

F(p)(A) = / wp x (dw)r](dx) 

wJ r (X, X^-(x))(dw)\ T](dx) + [ If wSo(dw)) rj(dx) 



drj 



+ 



^-{x)rj(dx) 
drj 

u{A v ) = v(A). 



A, 
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Moreover by taking Ei = En > 0} and E 2 = En {%{x) = 0} we get 

H{p\p l ®f 1 )+H{p 1 \Q{X))+I x {i 1 ) = H(p\Q(X) eg rj) 

= I H( Px (-)\Q(X)) V (dx) 



7.2 lid weighted bootstrap 

The following 

Lemma 7.4. Q is continuous. 

is the main argument in the proof of Theorem 2.2 

7.2.1 Proof of Lemma 7.4 

We consider a sequence of probability measures (p n ) n >i such that p n -4 p 

and a sequence of positive numbers m„ — >■ m > and prove that Q{p n ,m n ) — 5- 
^(/o, m). It is not difficult to show that for every / e Cb(R* + ), every p <E W 1 (K+) 
and every m > we have 



So /k+ xg(p n ,m n )(dx) = / R+ ^p n (dx) -> i / R+ xp(cfe) = J R+ m)(dx). 
Hence, according to Theorem 7.11 in [26] we are left to prove that for every 
/ <= Cb(M^) we have 



Since p n -4 p we have p n A p hence for every e > there exists a compact 




XH{u\ri), 



hence if (i/) < Ainf^g^ H(v\ij). 



□ 
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K e C R* + such that sup n>1 {p n {K^) 7 p(K^)} < e. Thus 



f(x)g(p n ,m n )(dx) 



f(—)p n (dx) 



f(x)G(p,m)(dx] 



< 



< 



K c 



f( — )p n (dx) 



f(-)p(dx) 



f(-)p(dx) 



/( — )-/(-) 

m„ m 



x 



p n (dx) + 



■2e\\f\\ 



f(-)p n (dx) 



f(~)p(dx) 
K m 



2e 



(7.4) 



Since / is continuous it is uniformly continuous over compact subsets of MJJ_ so 
there is an N such that for every n > N 



sup 



/(—)"/(-) 

rrin m 



< e. 



By taking No large enough we can make the second term in (7.4) as small as 
desired, which concludes the proof. □ 



7.2.2 Proof of Theorem 2.2 

According to [27] , that all the exponential moments of £ are finite is a necessary 
and sufficient condition for i Yn=i ^° sa tisfy an LDP on W 1 (K+) endowed 
with Wi t \.\ with good rate function H( ■ \p). Next, since 

G : W\R + ) -> K+ 

v ^ /«+ xv (dx) 

is continuous when W 1 (R+) is furnished with the W\ distance we obtain that 
(n ^y%i V7 Yh=i satisfies an LDP on the product space W 1 (K+) x K + 

with good rate function 



R{p,x) 



H{p\pi) if J K+ up(du) = x 
+oo otherwise. 



Since A*(0) = oo we have R(p, 0) = oo. Hence, according to Lemma 4.1.5 in [8], 
an LDP for J^=i ELi *i)n>i holds on W 1 (K+) x K+ with the same 

rate function R. Since Q is continuous and 



n 1 n 1 n 

7 7 ^ ^ 1 77 ' 77 ^ 



i = l 



i = l 
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(<S n ) n >i obeys an LDP on W 1 (IR+) due to the contraction principle, see Theo- 
rem 4.2.1 in [8]. Finally, the announced result follows from Lemma 4.1.5 in [8] 
since for every n > 1 S n <E (R+) and the latter is a closed subset of W 1 (M + ). 
□ 

7.2.3 Proof of Corollary 2.3 

Let v S Mi(E) be such that v <C fJ. for otherwise we already know that tC(v; fi) = 
+oo. We have 



K{v;ti) = inf \H(p\ Pl ®^+ M{H(g(-, Pl m 

= inf inf {ff(p| / > 1 ® M ) + fT(p 1 |0(m J O} 

m>0 p I :F(p x ®/j) = j/ 

= inf inf {H (p\G(m, £) ® /i)} 

m>0 p I :F(p :E (g)p)=f 

= mf inf ( / i?(fe|£(m, £))/«} 

(ft/ 

m>0 7s U/i 



> inf / A^(m— (x))n{dx) 



where, to establish the last inequality and characterize the equality, we proceed 
as in the proof of Lemma 1.1. □ 



7.3 The multivariate hypergeometric bootstrap 

Since the W™'s take values in the finite set {0, . . . ,K} the proof of this LDP 
closely follows the proof of Sanov's Theorem for finite alphabets as exposed in 
Section 2.1.1 in [8] . This is the reason why we shall not give all the details of 
the proof. The main step is to notice that for every (u>™, . . . , u>") € {0, ... , K} n 
such that Y^i=i w ? = n iiwe introduce 

1 n i n 

(u n (i), . . . , u n (K)) = (-J2 M 1 ). ■ • ■ > - E M*)) 

n £ — ' * n £ — ' 

i=l t=l 

we get J2k=i kv n (k) = 1 and 

/ -i i \nu n (k) 

p(w? = <, . . . , w„ n = <) = i?(n)nf =1 (c| (-) fe (i - ^) K - fe J (7-5) 

with i log i?(n) — > 0. For every integer n > 1, every f G A/j L ({0, . . . , if}) 
of the form ( — ,...,—) with k\,...,kx integers there exists T n (v) vectors 
(w?, . . . , O G™{0, . . . , K} n such that i/ = i £™=i S w? with 

(n + i)-^ e -" HM < T n {v) < e - nH ^ (7.6) 
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where H(v) = — J3 i=1 log v(i), see Lemma 2.1.8 in [8]. Hence, combining 
(7.5) and (7.6) for every integer n > 1, every ^ £ Mx({0, . . . ,K}) of the form 
(tT' • ■ • ' if ) with k i'---> k K integers we get 

^(nJe-^MSK*,*)) < P( 5 " = i/) < fl a (n) e -" ir < , 'l !8 < Jr '*» 

with i logi?i i 2(«) — > 0. The proof then follows as in Theorem 2.1.10 in [8] until 
its conclusion. □ 



7.4 A bootstrap generated from deterministic weights 

7.4.1 Proof of Corollary 2.5 

First we consider a > 0. Let v £ Mi(E) be such that A1 ~ (1 ~ a)iy £ Mi(E). Then 
p £ A^J(M+ x E) defined by p\ = 7 while the regular conditional distribution 
of its second marginal given the first is 

/i(da;) - (1 - a)v(dx) 
p_^{dx) = vidx) and po(dx) = 

is the only element of A^}(M+ x E) that satisfies F(p) = v,p\ = 7 and P2 = p- 
Since for this particular p we have 

H(p\pi <g> p 2 ) = (1 - + 

we obtain an upper-bound on /C as announced. To prove the reverse inequality 
let v £ Mx(E) be such that K,{v; p) < 00 for otherwise the announced result 
trivially holds. Necessarily there exists a p £ A^}(M+ x E) such that F{p) = 
f, px =7 an d P2 = These conditions are only met by the probability measure 
yO introduced above. In particular ^ t ~( 1 ~ a )' y m ust be a probability and the reverse 
inequality holds. 

If a = then 7 = <5i so the only v £ Mi(E) such that there exists p £ 
A^i(R_|- x E) such that F(p) = v and P2 = p is p and necessarily p = 8\ ® p. 
The announced result follows. □ 

7.4.2 Proof of Corollary 2.6 

First we consider a > 0. Let v £ Mi(E) be such that inf^ e £ v C) < 00 holds. 
Then for every C € £„ we have C ~ (1 ~ a)u £ Mi(E) and peM|(l + xS) defined 
by Pi = 7 and 

C(<fe) - (1 - a)i/(<£r) 
p_^(dx) — vidx) and po{dx) = 

satisfies -F(p) = u, pi = 7 and P2 = C hence /C(^; /i) < U{v, £) and the announced 
upper bound on /C follows. To prove the reverse inequality let us assume that 
K.{v;p) < 00. Then there exists a p £ .M}(M+ x E) such that F(p) = v 
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and p\ = 7. Necessarily p 1 (dx) = v{dx) and P2 is such that po(dx) = 
C(rf*)-(i-«M<faO e Mi(E) ^ Thug Ev ig non _ empty and 

H(p\p 1 ®p 2 )+I w (p 1 ) + I x (p 2 ) 
= (1 - a)g(i/lp3) + off ( P2 ~ ( * ~ |/*) J x (pa) 
> inf U(v,p 2 ) 

If a = then 7 = #1 and for every v G Mi(E) there is only one p G ,M;[(IR+ x E) 
such that -F(p) = f which is p = Si ® 1/ and the announced result immediately 
follows. □ 

7.5 The fc-blocks bootstrap 

7.5.1 Proof of Theorem 2.5 

Let us apply Corollary 2.1 to C n = ^ELi^r^i",-^™^ J- We obtain the 
desired result using the contraction principle on the map 

H : Mi(E fc ) -> Mi(S) 

pW h> W(p«) = ilt 1 p? ) 
which is continuous because it is Lipschitz. □ 

7.5.2 Proof of Corollary 2.7 

Let v G Mi(S). Since ~H{v® k ) = v and in this particular case p (fc) = p® fe we 
get 

K>;M (fe) ) < \n{y m \p® k ) 

k 

= H(v\n). 

On the other hand for every pw such that H(p^) = v we have 

i=l i=l 
since is convex. To conclude the proof just notice that 

> H(®LA k) \n m ) 

i=l 

a 



36 



A Topological properties of (.M}(M+ x £), A) and 
(M 1 l (R + ),W l ) 

A.l (M\(R+ x S), A) is a Polish space 
A.l.l (M\(R+ x £),A) is complete 

Let (p n ) n >i be a Cauchy sequence of elements of (A^}(IR+ x £), A). In particular 
(Pi )n>i is a Cauchy sequence of elements of (M*(R + ), W{) which is a complete 
space (see e.g. [4]) so there exists a pi G M\{R + ) such that Wi(pi,p\) — > 0. 
Furthermore (p n ) n >i is a Cauchy sequence of elements of (Mi(R+ x Y,),[3bl,s) 
hence there exists a 7 G Aii(R+ x S) such that pBL,s{p n ,7) — > 0. Necessarily 
we have 71 = p\ hence A(p™, 7) — > 0. 



A.1.2 (Ml(M+ x S),A) is separable 

Let R and E be dense countable subsets of and £ respectively. It is sufficient 
to prove that 



for every 1 < i < n, u™ G R and v™ G 



is dense in A/j 1 x E) endowed with the distance A. Indeed, since A^}(M+ x £) 
is a closed subset of A/' 1 1 (M+ x E) the announced claim follows. So let p G 
A/' 1 1 (R+ xE). In particular L xpi(dx) < 00. Lets us denote by (Xl, Yi), . . . , (X„, Y„), . . . 
a sequence of independent random variables with common distribution p. Ac- 
cording to Varadarajan's Lemma 

1 ™ 

— > S/ x ™ Y n ) ~> /° almost surely (A.l) 
n ' v 1 ' ' ' 

1 

and according to the strong law of Large Numbers 
1 n f 

— S^XJ 1 -* / xpi(dx) almost surely (A.2) 

n s V 

So there exists a family ((a;", y")i<i<n)n>i of elements of M+ x E such that 



n 1 n f 



xpi{dx). 



Since Rx E is dense in R+ x E for every n > 1 and every 1 < i < n there exists 
«, v") e Rx E such that max{|< - xf |, efe«, j/f )} < 2"". Clearly, 

i=l i=l 
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hence ^ + (i Yh=i S (u?,v?)>P) ~^ °- Silice both ^i." 5 and metrize 
the weak convergence topology we have that 

1 - 

pBL.s(-^2S(u?,v?),P) -^O. 
i=l 

Furthermore we have — X)"=i ^u" Pi and since 

1 n i n 1 n 

n * — ' n * — ' n * — ' 

i— 1 z— 1 z— 1 

< 2"™ 

we obtain that ^ X)"=i u ™ ~~ ^ Jr + xpi(dx) which together with the latter weak 
convergence ensure that i X)"=i ~^ /°i> according to Theorem 7.11 in [26]. 

A. 2 An approximation result on (M 1 1 (IR + ), Wi). 

Lemma A.l. For every p S Mj L (R+) i/iere exists for every n > 1 an elements 
p" of 



ve Af 1 1 (M+), 3 (xi,...,a; n ) G 0R+)V 



i=l J 



smc/i i/iai i/ n — £ p. 

Proof By the same kind of argument as in the separability proof above one can 
construct a sequence (i J27=i °~u n )n>i such that i X)™=i ~^ /°- I 11 particular 
— X)™=i u ? ~* !• So we onr y need to modify the Uj's in such a way that for every 
n > 1 their total sum equals n. For a fixed n we have three possibilities: 

• ^ Sj=i u » = ?1 we take w,™ = Uj for every 1 < i < n. 

• H Sr=i Ui > n w ^ look at the itj's as the occupation masses of n cells by a 
mass Uj each. We pick uniformly at random the excess of mass until we get new 
occupation masses u>™, . . . , w n such that $3 i=1 w? = 

• If Y2i=i ui < n again we look at the Mi's as the occupation masses and add 
mass uniformly at random into the n cells until they contain a total mass of n. 
We call u>™, . . . , w n the final occupation masses. 
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In all the cases considered above we have 



n 



i=l 



H- 0. 



- n 1 n ( 1 n i n 1 

n 

< sup 

ll/lli<I 
1 ™ 

»=i 

= i E 

»=i 
1 " 

i E 



□ 
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